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Abstract 

ROBART  111  is  intended  as  an  advanced  demonstration 
platform  for  non-lethal  tactical  response,  extending  the 
concepts  of  reflexive  teleoperation  into  the  realm  of 
coordinated  weapons  control  (i.e.,  sensor-aided  control 
of  mobility,  camera,  and  weapon  functions)  in  law 
enforcement  and  urban  warfare  scenarios.  A  rich  mix  of 
ultrasonic  and  optical  proximity’  and  range  sensors 
facilitates  remote  operation  in  unstructured  and 
unexplored  buildings  with  minimal  operator  oversight. 
Supervised  autonomous  navigation  and  mapping  of 
interior  spaces  is  significantly  enhanced  by  an  innovative 
algorithm  which  exploits  the  fact  that  the  majority  of 
man-made  structures  are  characterized  by  (but  not 
limited  to)  parallel  and  orthogonal  walls.  This  paper 
presents  a  brief  overview  of  the  advanced  telereflexive 
man-machine  interface  and  its  associated  “human- 
centered  mapping  ”  strategy’. 

1.  Background 

From  a  navigational  perspective,  the  type  of  control 
strategy  employed  on  a  mobile  platform  runs  the  full 
spectrum  defined  by  teleoperated  at  the  low  end  through 
frilly  autonomous  at  the  upper  extreme.  A  teleoperated 
machine  of  the  lowest  order  has  no  onboard  intelligence 
and  blindly  executes  the  drive  and  steering  commands 
sent  down  in  real-time  by  a  remote  operator.  A  fully 
autonomous  mobile  platform,  on  the  other  hand,  keeps 
track  of  its  position  and  orientation  and  typically  uses 
some  type  of  world  modeling  scheme  to  represent  the 
location  of  perceived  objects  in  its  surroundings.  A  very 
common  approach  is  to  employ  a  statistical  certainty-grid 
representation  [1],  where  each  cell  in  the  grid 
corresponds  to  a  particular  “unit  square”  of  floor  space. 
The  numerical  value  assigned  to  each  cell  represents  the 
probability  that  its  associated  location  in  the  building  is 
occupied  by  some  object,  with  a  value  of  zero  indicating 
free  space  (i.e.,  no  obstacles  present). 


The  existence  of  an  absolute  world  model  allows  for 
automatic  path  planning,  and  subsequent  route  revisions 
in  the  event  a  new  obstacle  is  encountered. 
Unfortunately,  however,  the  autonomous  execution  of 
indoor  paths  generally  requires  a  priori  knowledge  of  the 
floorplan  of  the  operating  environment,  and  in  all  cases 
the  robot  must  maintain  an  accurate  awareness  of  its 
position  and  orientation.  Accordingly,  traditional 
autonomous  navigation  techniques  are  of  limited  utility 
for  applications  where  a  requirement  exists  to  enter 
previously  unexplored  structures  of  opportunity  as  the 
need  arises. 

Teleoperated  systems,  on  the  other  hand,  permit  remote 
operation  in  such  unknown  environments,  but 
conventionally  place  unacceptable  demands  on  the 
operator.  For  example,  simply  driving  a  teleoperated 
platform  using  vehicle-based  video  feedback  is  no  trivial 
matter,  and  can  be  stressful  and  fatiguing  even  under  very 
favorable  conditions.  If  a  remote  operator  has  to  master 
simultaneous  inputs  for  drive,  steering,  camera,  and 
weapons  control,  the  chances  of  successfully  performing 
coordinated  actions  in  a  timely  fashion  are  minimal. 

Easing  the  driving  burden  on  the  operator  was  a  major 
force  behind  the  development  of  the  reflexive 
teleoperated  control  scheme  employed  on  ROBART  II 
[2,  3],  a  prototype  security  robot  capable  of  both 
teleoperated  and  autonomous  operation.  The  robot’s 
numerous  collision-avoidance  sensors,  originally 
intended  to  provide  an  envelope  of  protection  during 
autonomous  transit,  were  also  called  into  play  during 
manual  operation  to  greatly  minimize  the  possibility  of 
operator  error.  The  commanded  velocity  and  direction  of 
the  platform  was  altered  by  the  onboard  processors  to 
keep  the  robot  traveling  at  a  safe  speed  and  preclude 
running  into  obstructions.  Work  on  ROBART  III  (figure 
|1|)  now  extends  this  reflexive -teleoperation  concept  into 
the  realm  of  sensor-assisted  camera  and  weapon  control 
for  indoor  tactical  systems. 
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velocity  is  increased  one  increment.  Clicking  on  either 
the  right-  or  left-turn  arrows  imposes  a  differential  turn 
on  the  forward  velocity,  speeding  up  one  wheel  and 
slowing  down  the  other.  The  more  times  a  turn  arrow  is 
clicked,  the  bigger  the  differential  and  hence  the  faster 
the  rate  of  turn.  If  the  forward  (or  reverse)  speed  is  zero 
(i.e.,  platform  stopped),  clicking  a  turn  button  causes  the 
robot  to  pivot  in  place. 


Figure  2.  Navigation  Control  Screen,  showing  the  high- 
level  driving  icons  surrounding  the  Map  Window  (lower 
left  corner).  The  robot  has  been  instructed  to  enter  the 
next  door  encountered  on  the  left. 


Figure  1.  ROB  ART  III  is  a  laboratory  prototype 
supporting  the  development  of  enhanced  telereflection 
control  strategies  for  tactical  response  robots. 


2.  Man-Machine  Interface 


A  very  simplistic  graphical  user  interface  (GUI)  has  been 
implemented  under  Visual  Basic  to  support  the 
development  and  diagnostic  needs  of  this  technology- 
base  effort  (Figure  2).  For  purposes  of  this  discussion, 


the  man-machine  interface  issues  can  be  subdivided  into 
three  general  categories:  1)  mobility  control,  2)  camera 
control,  and  3)  non-lethal  weapon  control. 


2.1  Mobility  Control 

The  Mobility  Control  Window  (lower  right  corner  of  the 
screen)  provides  a  convenient  means  for  the  operator  to 
set  the  desired  speed,  and  if  necessary,  manually  change 
the  platform’s  heading.  Each  time  the  operator  clicks  on 
the  forward  arrow  button,  for  example,  the  platform’s 


Once  the  platform  is  set  in  motion,  the  operator  can 
easily  control  its  subsequent  actions  by  clicking  on 
special  behavioral  icons  depicted  on  the  navigation 
display.  For  example,  selecting  a  wall-following  icon 
causes  the  platform  to  enter  wall-following  mode, 
maintaining  its  current  lateral  offset  from  the  indicated 
wall  using  side-looking  sonar.  The  wall-following  icons 
are  implemented  as  long  vertical  command  buttons 
situated  on  either  side  of  the  Map  Window  in  the  lower 
left  corner.  The  nine  dots  displayed  in  front  of  the 
rectangular  robot  icon  at  the  bottom  of  the  map  indicate 
the  measured  range  to  perceived  objects  in  the  path. 


Two  additional  wall-segment  icons  are  seen  above  the 
map  in  the  form  of  short-length  horizontal  command 
buttons.  The  open  spaces  between  these  graphical 
depictions  of  wall  structures  represent  three  potential 
doorways',  one  directly  ahead  of  the  robot  and  one  on 
either  side.  By  clicking  in  one  of  these  doorway  icons, 
the  robot  is  instructed  to  seek  out  and  enter  the  next 
encountered  location  of  that  type  of  door  along  its  current 
path.  For  the  example  illustrated  in  Figure  2  the 
platform  is  looking  for  a  door  off  to  the  left,  as  indicated 
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by  the  highlight  box  shown  in  the  selected  dooi~way  icon, 
and  the  associated  text  displayed  in  the  System  Status 
Window  above  the  map. 


The  primary  mobility  controls  shown  in  Figure  2 
mimicked  on  a  stand-alone  hand-held  pendent  (Figure 


are 


employing  an  array  of  capacitive  touch-sensor  icons, 
based  on  the  Quantum  Research  QProx  E6S2  matrix 
decoder.  A  high-resolution  2.5-inch  color  LCD  monitor 
provides  video  output,  in  addition  to  selected  status 
information  overlaid  at  the  top  of  the  screen.  A  miniature 
motor-driven  eccentric  (as  is  commonly  found  in 
vibrating  pagers)  is  mounted  inside  the  enclosure  to 
provide  tactile  motion  feedback  to  the  user  [4].  The 
speed  of  this  motor  (and  hence  the  vibration  of  the  case) 
is  varied  in  direct  proportion  to  the  velocity  of  the  remote 
platform. 


Figure  3.  A  capacitive  touch-panel  interface  on  the 
hand-held  pendent  mimics  the  drive  icons  shown  in 
Figure  2, 


2.2  Camera  Control 

Manual  control  of  the  ROBART’s  head-mounted  camera 
can  be  accomplished  using  the  slider  and  button  controls 
within  the  Head  Pan  Control  Window  on  the  right  side  of 
the  display  screen.  In  addition,  computer-aided  camera 
pan  is  provided  to  support  the  three  system  functionalities 
of  platform  mobility,  intruder  assessment,  and  weapon 
tracking.  For  mobility,  the  camera-pan  commands  are 
embedded  within  the  “seek-door”  behaviors.  If  the  robot 
is  instructed  to  enter  the  next  door  on  the  right,  for 
example,  the  camera  immediately  turns  45  degrees  right 
of  center  to  acknowledge  the  behavior  request  and 
provide  a  better  view  of  the  doorway  detection  process. 
As  soon  as  the  door  is  detected  and  the  penetration 
behavior  invoked,  the  camera  pans  to  compensate  for  the 


platform’s  rate  of  turn  in  order  to  keep  the  door  opening 
in  the  center  of  its  field-of-view. 

The  intruder  detection  and  assessment  algorithms  operate 
upon  the  output  from  the  video  motion  detection  (VMD) 
system  and  a  360-degree  array  of  passive-infrared  (PIR) 
sensors  configured  as  a  collar  just  below  the  head.  The 
PIR  data  is  used  to  pan  the  surveillance  camera  to  the 
center  of  any  zone  with  suspected  intruder  activity.  The 
VMD  output  is  then  used  to  track  and  keep  the  intruder  in 
the  center  of  the  visual  field,  using  a  combination  of 
robot  head  and  body  movement. 

Whenever  the  head  reaches  its  maximum  pan  limit  (+100 
degrees)  relative  to  the  robot,  the  mobility  base  will  pivot 
in  place  towards  the  target.  The  head  meanwhile  moves 
at  the  same  speed  in  the  opposite  direction  to  keep  the 
primary  target  in  the  center  of  the  visual  field.  This 
coordinated  action  provides  the  robot  with  unlimited  (i.e., 
continuous  360-degree)  pan  coverage. 

Automated  camera  pan  for  weapon  tracking  is  treated  in 
the  next  section. 


2.3  Non- Lethal  Weapon  Control 


The  principle  non-lethal  response  system  incorporated  on 
ROBART  III  is  a  six-barreled  pneumatically-powered 
Gatling-gun  (Figure  4|  capable  of  firing  3/16-inch- 


diameter  simulated  tranquilizer  darts  or  plastic  bullets. 
Projectiles  are  expelled  at  a  high  velocity  from  12-inch 
barrels  by  a  release  of  compressed  air  from  a  pressurized 
accumulator  at  the  rear  of  the  gun  assembly.  The  main 
air  bottle  is  automatically  recharged  by  a  small  12-volt 
reciprocating  compressor  mounted  in  the  robot’s  base. 


Figure  4.  A  six-barrel  pneumatic  tranquilizer  gun  is  used 
to  demonstrate  computer-assisted  control  of  a  non-lethal 
weapon. 
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The  operator  specifies  what  type  of  control  strategy  (i.e., 
manual  or  automatic)  to  use  when  entering  weapon¬ 
tracking  mode  by  clicking  on  the  appropriate  option  in 
the  Track  Mode  Window  shown  in  the  bottom-right 
corner  of  Figure  5  In  manual  mode,  the  firing  decision 
is  made  by  the  operator.  A  5-milliwatt  670-nanometer 
visible-red  laser  sight  facilitates  manual  training  of  the 
weapon  using  video  from  the  head-mounted  surveillance 
camera.  The  operator  can  slave  the  surveillance-camera 
pan  to  the  weapon  pan  axis  by  clicking  on  the  “Head” 
option  in  the  Slave  Window  (just  below  the  System  Status 
Window,  upper  left  comer).  The  mobility  base  can  also 
be  slaved,  so  the  robot  turns  to  face  the  direction  the 
weapon  is  aimed.  If  a  forward  drive  speed  is  entered  at 
this  point,  the  operator  merely  has  to  keep  the  weapon 
trained  on  the  intruder,  and  the  robot  will  automatically 
give  chase. 


Figure  5.  Interim  control  and  diagnostic  screen  used 
during  development  of  the  computer-assisted-weapon- 
control  software  on  ROBART  III. 


In  automatic  mode,  ROBART  III  is  responsible  for 
making  the  firing  decision,  contingent  upon  a  confirmed 
target  solution  stabilized  for  a  pre-determined  time 
interval,  and  pre-authorization  from  the  operator. 
Azimuthal  and  elevation  information  from  the  VMD  is 
available  to  the  right-shoulder  pan-and-tilt  controller  for 
purposes  of  automated  weapon  positioning.  When 
weapon-tracking  is  activated  in  automatic  mode,  the 
robot  centers  its  head  and  turns  to  face  toward  the  current 
threat.  The  mobility  base  then  becomes  stationary  while 
the  weapon  begins  tracking  the  target. 


3.  Human-Centered  Mapping 

The  exploration  and  mapping  of  unknown  structures 
benefits  significantly  when  the  interpretation  of  raw 
sensor  data  is  augmented  by  simultaneous  supervisory 
input  from  the  human  operator.  A  human-centered 
mapping  strategy  has  been  developed  to  ensure  valid 
first-time  interpretation  of  navigational  landmarks  as  the 
robot  builds  its  world  model  (currently  on  an  external 
RF-linked  desktop  PC).  In  a  nutshell,  the  robot  can  enter 
and  explore  an  unknown  space,  building  a  valid  model 
representation  on  the  fly,  while  dynamically  re¬ 
referencing  itself  in  the  process  to  null  out  accumulated 
dead-reckoning  errors. 

Upon  first  entering  a  previously  unexplored  building,  the 
operator  guides  the  robot  using  typical  commands  like: 
“follow  the  wall  on  your  left,”  and  “enter  the  next 
doorway  on  the  left.”  Such  high-level  direction  is 
provided  by  clicking  on  screen  icons  as  previously 
described.  With  this  minimal  operator  input,  the  robot  in 
this  example  doesn’t  just  think  it  sees  a  wall,  it  knows  it 
sees  a  wall.  In  other  words,  in  addition  to  directing  the 
robot’s  immediate  behavior,  these  same  commands  also 
provide  valuable  information  to  the  world  modeling 
algorithm.  The  end  result  of  such  an  approach  is  a  much 
faster  and  more  accurate  generation  of  object 
representations  (relative  to  conventional  sensor-only  data 
collections),  particularly  valuable  when  there  is  no  a 
priori  information  available  to  the  system. 

The  world  model  is  first  initialized  as  a  two-dimensional 
dynamic  array  with  all  cells  marked  as  unknown.  (An 
unknown  cell  is  treated  as  potentially  traversable,  but 
more  likely  to  be  occupied  than  confirmed  free  space)  If 
some  specific  subset  of  the  current  sonar  data  can  be 
positively  identified  from  the  outset  as  a  wall-like 
structure,  it  can  be  unambiguously  modeled  as  a 
confirmed  wall  without  the  need  for  statistical 
representation.  This  makes  the  resulting  world 
representation  much  less  ambiguous  and  therefore  less 
subject  to  error. 

In  support  of  this  objective,  ROBART  III  has  been 
mechanically  and  electronically  equipped  specifically  to 
support  supervised  operation  in  previously  unexplored 
interior  structures.  Two  self-contained  Electro 
Corporation  piezoelectric  PCUC-series  ultrasonic  sensors 
operating  at  215  KHz  are  used  to  generate  range  data  for 
the  wall-following  algorithm.  (These  sonar  sensors 
operate  at  a  much  higher  frequency  than  the  49.4-KHz 
Polaroid  sensors  used  for  collision  avoidance,  so  there 
are  no  problems  associated  with  crosstalk  from 
simultaneous  operation.) 
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4.  Orthogonal  Navigation 

The  Achilles  Heel  of  any  world-modeling  scheme, 
however,  is  accurate  positional  referencing  in  real-time 
by  the  moving  platform.  Since  all  sensor  data  is  taken 
relative  to  the  robot’s  location  and  orientation,  the 
accuracy  (and  usefulness)  of  the  model  quickly  degrades 
as  the  robot  becomes  disoriented.  While  wall  following 
is  a  very  powerful  tool  in  and  of  itself  for  determining  the 
relative  offset  and  heading  of  the  robot,  conventional 
schemes  normally  assume  some  a  priori  information 
about  the  wall  in  the  first  place  to  facilitate  its  utility  as  a 
navigational  reference.  In  short,  a  relative  fix  with 
respect  to  an  unknown  entity  does  not  yield  an 
unambiguous  absolute  solution,  for  obvious  reasons. 

ROBART  III  uses  a  new  and  innovative  world  modeling 
technique  that  requires  no  such  a  priori  information. 
This  navigation  scheme,  called  orthogonal  navigation,  or 
“Ortho-Nav,  ”  exploits  the  orthogonal  nature  of  most 
building  structures  where  walls  are  parallel  and 
connecting  hallways  and  doors  are  orthogonal.  Ortho- 
Nav  also  uses  the  input  from  a  magnetic  compass  to 
address  the  issue  of  absolute  wall  orientation.  The 
accuracy  of  the  compass  need  be  only  good  enough  to 
resolve  the  ambiguity  of  which  of  four  possible  wall 
orientations  the  robot  has  encountered.  This  information 
is  stored  in  the  model  in  conjunction  with  the  wall 
representation  (i.e.,  wall  segment  running  north-south,  or 
wall  segment  running  east-west),  in  arbitrary  building 
coordinates.  The  precise  heading  of  the  vehicle  (in 
building  coordinates)  is  then  mathematically  derived 
using  sonar  data  taken  from  the  wall  surface  as  the  robot 
moves. 


A  typical  wall-following  routine  uses  a  ranging  sensor  to 
maintain  a  particular  distance  from  a  planar  object  (wall) 
on  one  or  both  sides.  Due  to  sensor  inaccuracies  and  the 
accumulation  of  errors  inherent  in  odometry,  the  range 
data  will  appear  to  drift  toward  or  away  from  the  robot, 
resulting  in  a  wall  plot  that  is  skewed  or  perhaps  even 
curved.  These  errors  can  be  mitigated  by  assuming  that 
the  wall  is  straight  and  immovable,  and  any  perceived 
undulations  in  the  sonar  data  plot  in  actuality  represent 
irregular  motion  of  the  robot.  Armed  with  this  heuristic, 
both  the  lateral  offset  and  heading  of  the  robot  can  be 
dynamically  corrected,  even  while  the  world  model  is 
still  being  generated. 


In  order  for  this  system  to  work  properly,  the  robot  must 
follow  a  reasonably  planar  wall  surface  rather  than  just 
blindly  reacting  to  whatever  clutter  is  nearby.  This  is 
where  the  human-centered  aspect  of  the  scheme  comes 
into  play.  By  way  of  example,  when  the  robot  enters  an 
unknown  space  under  telereflexive  control  as  illustrated 
in  Figure  6|  the  operator  examines  the  video  and  informs 


the  robot  there  is  a  wall  it  can  follow  on  the  left  side.  In 
addition,  the  operator  also  clicks  on  the  left  doorway  icon 
(as  illustrated  earlier  in  Figure  2  i  to  further  instruct  the 
robot  to  find  and  enter  the  next  doorway  on  the  left.  The 
onboard  computer  then  begins  acquiring  range  data  from 
the  appropriate  sensors.  When  enough  points  have  been 
accumulated  for  a  fit  (subject  to  a  quality-of-fit-criteria), 
the  resulting  line  is  examined  to  determine  its  orientation. 


Figure  6.  Initial  view  of  an  interior  space  as  seen  from 
the  robot’s  onboard  surveillance  camera,  revealing  a 
clean  wall  for  following  on  the  immediate  left. 


The  majority  of  buildings  are  laid  out  such  that  all  walls 
are  either  parallel  or  orthogonal  to  one  another,  so  the 
orientation  of  the  line  is  snapped  to  0°,  90°,  180°,  or  270° 
in  arbitrary  building  coordinates.  The  robot’s  heading  is 
then  reset  to  this  same  value.  Once  the  initial  location  of 
the  wall  has  been  established,  an  infinitely  \ong  potential- 


wall  representation  is  entered  into  the  model  (Figure  7 


Figure  7.  After  obtaining  the  first  wall  fit,  a  potential 
wall  is  created  and  indexed  to  the  cardinal  heading  which 
most  closely  matches  the  magnetic  compass  reading. 
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As  the  robot  continues  to  follow  the  actual  wall  getting 
valid  line  fits,  it  incrementally  converts  the  potential  wall 
to  a  con  firmed  wall. 

As  previously  discussed,  the  robot  can  now  correct  its 
lateral  position  by  using  the  wall  as  a  reference.  For  the 
situation  shown  above,  given  that  the  X  coordinate  of  the 
wall  is  Wx  and  the  current  range  to  the  wall  is  r,  the 
robot’s  X  coordinate  is  given  by  Rx  =  Wx  +  r.  Similarly, 
the  robot’s  current  heading  can  be  dynamically  corrected 
by  subtracting  the  difference  between  the  orientation  of 
the  current  wall  fit  and  the  current  wall  orientation  from 
the  robot’s  current  heading.  This  is  given  by  the 
following  equation: 

R,=R,-(F,-Wt) 

where: 

Re  =  the  robot’s  current  heading 

Fg  =  the  orientation  of  the  current  wall  fit 

Wg  =  the  orientation  of  the  current  wall 


Confirmed  Wall 


Potential  Wall 


Confirmed  Wall  - 


w 


Figure  8.  As  the  robot  enters  the  found  doorway,  the 
modeling  algorithm  uses  the  side-sonar  range  information 
to  cut  an  appropriately-sized  opening  through  the  existing 
(shown  vertical)  confirmed-  and  potential-wall 
representations.  The  new  (horizontal)  potential  wall  is 
clipped  against  the  previously  constructed  potential  wall. 


The  robot  turns  left  (as  previously  instructed)  to  enter  the 
discovered  doorway,  using  the  ranging  sensors  on  both 
sides  to  determine  the  size  of  the  opening  that  must  be 


cut  in  the  wall  it  has  been  constructing  ([figure  8|,  to  form 
the  doorway  representation.  After  transitting  the 
doorway,  the  robot  next  detects  and  begins  to  follow  a 
wall  to  its  right.  Accordingly,  it  constructs  a  new 
potential  wall  and  snaps  it  perpendicular  to  the  previous 
model  entry.  Note  this  second  potential  wall  (shown 
horizontally)  is  semi-infinite,  in  that  it  is  clipped  against 
the  previously  constructed  potential  wall  (shown 
vertically). 


Whenever  the  robot  detects  a  new  potential  wall,  it 
compares  it  to  the  list  of  potential  and  confirmed  walls 
already  constructed.  If  the  new  wall  coincides  (within 
pre-specified  orientation  and  offset  tolerances)  with  a 
previously  modeled  wall,  the  range  data  is  snapped  to  the 
existing  representation,  rather  than  generating  a  new  one. 


5.  Conclusion 

This  paper  covers  the  implementation  of  a  prototype 
tactical/security  response  robot  capable  of  semi- 
autonomous  exploration  in  unknown  structures.  The 
system  is  able  to  confront  intruders  with  a  laser-sighted 
tranquilizer  dart  gun,  and  automatically  track  a  moving 
target  with  the  use  of  various  sensors.  A  human-centered 
mapping  scheme  ensures  more  accurate  first-time 
interpretation  of  navigational  landmarks  as  the  robot 
builds  its  world  model,  while  orthogonal  navigation 
exploits  the  fact  that  the  majority  of  man-made  structures 
are  characterized  by  parallel  and  orthogonal  walls. 
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